Method for forming 3D maxillofacial model by automatically segmenting medical image, automatic image segmentation and model formation server performing the same, and storage medium storing the same

ABSTRACT

Disclosed is a method for forming a three-dimensional (3D) model of skin and mandible by automatic medical image segmentation which is performed in an automatic image segmentation and model formation server. The method includes (a) receiving 3D medical image data that is a set of two-dimensional (2D) images for horizontal planes of a face, (b) obtaining a contrast histogram based on distribution of contrasts of the 3D medical image data, and segmenting the 3D medical image data for the face into multiple regions separated into at least one partial region based on the contrast histogram, (c) extracting only the face by removing portions other than the face from the multiple regions for the face, and extracting a skin region of the face, (d) extracting the mandible from each of the 2D images for the horizontal planes of the face through a 2D detailed segmentation technique using an active contour method based on a level set function, and (e) reconstructing the extracted skin region and mandible as the 3D model.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to and the benefit of Korean PatentApplication No. 2015-0132843, filed on Sep. 21, 2015, the disclosure ofwhich is incorporated herein by reference in its entirety.

BACKGROUND

1. Field of the Invention

The present invention relates to a technology for forming athree-dimensional (3D) model of skin and mandible by automatic imagesegmentation, and more particularly, to a method for forming a 3Dmaxillofacial model by automatic medical image segmentation, which mayrapidly and efficiently form a 3D model of skin and mandible bycombining a macroscopic 3D multi-segmentation technique and 2D detailedsegmentation technique, an automatic image segmentation and modelformation server performing the same, and a storage medium storing thesame.

2. Discussion of Related Art

The mandible is a bone that is mainly used to chew food and an importantportion to hold the contours of the face. In the mandible, orthognathicsurgery is carried out for mainly cosmetic purposes, and mandibularfracture surgery due to trauma and mandibular resection due to cancerare carried out. In a case of reconstructive surgery other than acosmetic purpose, when a surgery to replace the mandible using thefibula of a patient is carried out, soft tissue removal and ostectomyare carried out together with several departments such as dentistry,otolaryngology, surgery, and the like, and the reconstructive surgery isone of surgeries with a high level of difficulty which takes more than10 hours. As a method for increasing the success rate of such a surgerywith the high level of difficulty and reducing the operation time,three-dimensional (3D) virtual surgical planning techniques have beenrecently tried. In the prior art, a doctor directly performs a manualimage segmentation work for modeling a human body part that is requiredduring a surgery, and therefore a period of about a week or more to planthe surgery has been spent and such a manual modeling results in adecrease in interobserver agreement.

In addition, in Korea Patent Publication No. 10-2006-0028044 whichrelates to a 3D finite element modeling method using two-dimensional(2D) medical images and a storage medium, a method of performingcomputer simulation by configuring a 3D shape model has been disclosed.However, as the specific method thereof, an object and a background areseparated from the 2D medical images, the 2D images from which thebackground is separated are stacked, and then the outlines of thestacked images are connected to each other to form a 3D shape modelwhose inside is empty, and the 3D shape model whose volume is filledwith triangular pyramid-shaped tetrahedron elements is finally generatedby adjusting the accuracy of the 3D shape model, so that the 3D shapemodel is generated only through a process of separating the backgroundand the object, and therefore a difference between an actual structureof the object and the generated 3D shape model may occur. In this case,it may be unsuitable to establish a surgical plan, and therefore theabove-described problem still exists.

Therefore, there is a demand for an automatic segmentation based-3Dmodeling technique for efficient and stable surgical plan andsimulation.

PRIOR ART DOCUMENT Patent Document

-   [Patent Document 1] Korea Patent Publication No. 10-2006-0028044

SUMMARY OF THE INVENTION

The present invention is directed to a method for forming athree-dimensional (3D) maxillofacial model by automatic medical imagesegmentation, which may automatically segment the skin and mandible of apatient by combining a multi-segmentation method and a detailedsegmentation method using a level set function and generate a 3D model,and thereby may help to establish more accurate and efficient surgicalplan and obtain a uniform mandibular model of the patient, an automaticimage segmentation and model formation server performing the same, and astorage medium storing the same.

The present invention is also directed to a method for forming a 3Dmaxillofacial model by automatic medical image segmentation, which maybe performed at a high speed while overcoming difficulties in thesegmentation due to irregularities in boundaries caused by high and lowcontrasts, an automatic image segmentation and model formation serverperforming the same, and a storage medium storing the same.

According to an aspect of the present invention, there is provided amethod for forming a three-dimensional (3D) model of skin and mandibleby automatic medical image segmentation which is performed in anautomatic image segmentation and model formation server, including: (a)receiving 3D medical image data that is a set of two-dimensional (2D)images for horizontal planes of a face; (b) obtaining a contrasthistogram based on distribution of contrasts of the 3D medical imagedata, and segmenting the 3D medical image data for the face intomultiple regions separated into at least one partial region based on thecontrast histogram; (c) extracting only the face by removing portionsother than the face from the multiple regions for the face, andextracting a skin region of the face; (d) extracting the mandible fromeach of the 2D images for the horizontal planes of the face through a 2Ddetailed segmentation technique using an active contour method based ona level set function; and (e) reconstructing the extracted skin regionand mandible as the 3D model.

Preferably, the (b) obtaining of the contrast histogram may includedividing a range of the contrast into predetermined levels, andcalculating the number of pixels having the contrast corresponding toeach of the levels from the 3D medical image data for the face tothereby obtain the contrast histogram.

Preferably, the (b) obtaining of the contrast histogram may furtherinclude extracting, from the contrast histogram, a partial region havinga peak satisfying a specific criterion using an AGMC (adaptive globalmaximum clustering) technique.

Preferably, the (b) obtaining of the contrast histogram may furtherinclude segmenting the 3D medical image data into the multiple regionsbased on an average value of the contrast histogram of the partialregion.

Preferably, the (c) extracting of only the face and the skin regionthereof may include obtaining a face candidate region by binarizing themultiple regions according to label values of the multiple regions.

Preferably, the (c) extracting of only the face and the skin regionthereof may further include eroding the face candidate region using acircular structural element having a radius value set based on a size ofthe face candidate region or a preset radius value.

Preferably, the (c) extracting of only the face and the skin regionthereof may further include restricting an erosion region reaching froma center of the face candidate region to positions laterally away fromeach other by a preset length.

Preferably, the (c) extracting of only the face and the skin regionthereof may further include extracting a connection component having acommon portion with the erosion region, and expanding the erosion regionusing the circular structural element.

Preferably, the (d) extracting of the mandible may include selecting asample image from the 2D images, and extracting the mandible from thesample image that is a segmentation result for the sample image

Preferably, the (d) extracting of the mandible may further includesetting an initial contour of an image next to or prior to the sampleimage based on a contour for a segmentation result of the sample image.

Preferably, the (d) extracting of the mandible may further includeemphasizing a contrast of the mandible based on contrast informationabout the mandible inside the initial contour.

Preferably, the (d) extracting of the mandible may further includestopping movement of the contour when the initial contour moves andreaches a boundary of the mandible.

Preferably, the (d) extracting of the mandible may further include thecontour moving based on an average value of local contrasts of theinside and outside of the contour and a curvature of the contour.

Preferably, the (e) reconstructing of the extracted skin region andmandible may include reconstructing the extracted skin region andmandible as the 3D model using a surface rendering algorithm, andprocessing a surface of the 3D model using an HC-Laplacian algorithm

Preferably, the 3D medical image data may be CBCT (cone beam computedtomography) image data.

Preferably, the segmented multiple regions may be labeled according tocontrasts of the multiple regions.

According to another aspect of the present invention, there is providedan automatic image segmentation and model formation server whichperforms a method for forming a 3D maxillofacial model by automaticmedical image segmentation, including: an image data reception unit thatreceives 3D medical image data that is a set of 2D images for horizontalplanes of a face; a multi-region segmentation unit that obtains acontrast histogram based on distribution of contrasts of the 3D medicalimage data, and segments the 3D medical image data for the face intomultiple regions separated into at least one partial region based on thecontrast histogram; a purpose region segmentation unit that includes askin detailed segmentation module for extracting only the face byremoving portions other than the face from the multiple regions for theface and extracting a skin region of the face, and a mandible detailedsegmentation module for extracting the mandible from each of the 2Dimages for the horizontal planes of the face through a 2D detailedsegmentation technique using an active contour method based on a levelset function; and a 3D reconstruction unit that reconstructs theextracted skin region and mandible as the 3D model.

Preferably, the multi-region segmentation unit may obtain the contrasthistogram about the 3D medical image data, and segment the 3D medicalimage data into the multiple regions based on an average value of thecontrast histogram of a partial region having a peak satisfying aspecific criterion in the contrast histogram.

Preferably, the skin detailed segmentation module may obtain a facecandidate region from the multiple regions, erode the face candidateregion using a circular structural element based on a size of the facecandidate region, restrict an erosion region based on a length of theface candidate region, extract a connection component having a commonportion with the erosion region, and expand the erosion region using thecircular structural element.

Preferably, the mandible detailed segmentation module may segment indetail an image next to or prior to a sample image based on a contourfor a segmentation result of the sample image among the 2D images tothereby extract the mandible.

According to still another aspect of the present invention, there isprovided a recording medium that records a program for executing amethod for forming a 3D maxillofacial model by the above-describedautomatic medical image segmentation.

The method for forming the 3D maxillofacial model by automatic medicalimage segmentation according to the present invention may be implementedby a computer-readable code on a computer-readable recording medium. Thecomputer-readable recording medium includes all of recording devicesthat store computer-readable data.

For example, the computer-readable recording medium may be a ROM, a RAM,a CD-ROM, a magnetic tape, a hard disk, a floppy disk, a mobile storagedevice, a non-volatile memory (flash memory), an optical data storagedevice, and the like.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the presentinvention will become more apparent to those of ordinary skill in theart by describing in detail exemplary embodiments thereof with referenceto the accompanying drawings, in which:

FIG. 1 is a block diagram illustrating an automatic image segmentationand model formation server according to an embodiment of the presentinvention;

FIG. 2 is a flowchart illustrating a method for forming athree-dimensional (3D) model by automatic medical image segmentation,which is performed in the automatic image segmentation and modelformation server of FIG. 1;

FIGS. 3A-3C is an example of 3D medical image data;

FIGS. 4A-4B is an example of a contrast histogram;

FIGS. 5A-5B is an example of results of multi-region segmentation;

FIG. 6 is an example of a skin detailed segmentation process;

FIGS. 7A-7E is an example of an erosion and expansion operation;

FIG. 8 is an example of a process of extracting a mandible from a sampleimage;

FIG. 9 is an example of initial contour setting;

FIG. 10 is an example of two-dimensional (2D) detailed segmentationusing a level set function;

FIG. 11 is an example of segmentation by an active contour;

FIG. 12 is a diagram illustrating a relationship between a contour and alevel set function;

FIG. 13 is a diagram illustrating a phase change of a level set method;

FIG. 14 is an example of a target simultaneously having clear boundariesand blurry boundaries;

FIG. 15 is an example of a process of acquiring a contrast-enhancingimage of a target;

FIGS. 16A-16B is an example of an edge indicator function;

FIG. 17 is an example of a detailed segmentation process using a levelset function;

FIG. 18 is an example of 3D model reconstruction and smoothing results;

FIG. 19 is a diagram illustrating comparison between 2D images of aconventional segmentation method and a segmentation method according toan embodiment of the present invention; and

FIG. 20 is a diagram illustrating comparison between 3D models of aconventional segmentation method and a segmentation method according toan embodiment of the present invention.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Advantages and features of the present invention and a method forachieving the same will become explicit by referring to the exemplaryembodiments that are described in detail in the following with referenceto the accompanying drawings. However, the present invention is notlimited to the exemplary embodiments disclosed in the following andthus, may be configured in various forms. Here, the present exemplaryembodiments are provided to make the disclosure of the present inventionperfect and to completely inform those skilled in the art about thescope of the present invention. The present invention is defined by thescope of claims. Like numbers refer to like elements throughout thedescription of the figures. As used herein, the term “and/or” includesany and all combinations of one or more of the associated listed items.

Although the terms first, second, etc. may be used herein to describevarious elements, components, and/or sections, these elements,components, and/or sections should not be limited by these terms. Theseterms are only used to distinguish one element, component, or sectionfrom another element, component, or section. Thus, a first element,component, or section discussed below could be termed a second element,component, or section without departing from the teachings of thepresent invention.

In addition, reference characters (for example, a, b, c, etc.) relatedto steps are used for convenience of description, and are not intendedto describe the sequence of the steps. The steps may occur in differentsequences, as long as a specific sequence is not specifically describedin the context. That is, the steps may occur in a specified sequence,may occur simultaneously, or may be performed in the reverse sequence.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”,“comprising,”, “includes” and/or “including”, when used herein, specifythe presence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which this invention belongs. It will befurther understood that terms, such as those defined in commonly useddictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art andwill not be interpreted in an idealized or overly formal sense unlessexpressly so defined herein.

Also, if it is determined that a specific description of the related andnoticed functions or structures may obscure the gist of the presentinvention, the specific description will be omitted. In addition, termsare to be described later which may vary according to the custom orintention of users or operators as the terms defined in consideration offunctions in the present invention. Therefore, definition of the termsshould be made based on the contents throughout the specification.

FIG. 1 is a block diagram illustrating an automatic image segmentationand model formation server according to an embodiment of the presentinvention.

Referring to FIG. 1, an automatic image segmentation and model formationserver 100 according to an embodiment of the present invention includesan image data reception unit 110, a multi-region segmentation unit 120,a purpose region segmentation unit 130, a three-dimensional (3D)reconstruction unit 140, and a control unit 150, and the purpose regionsegmentation unit 130 includes a skin detailed segmentation module 131and a mandible detailed segmentation module 132.

The image data reception unit 110 receives 3D medial image data that isa set of two-dimensional (2D) images. According to an embodiment of thepresent invention, the image data reception unit 110 may receive a 3DCBCT (cone beam computed tomography) image, and for example, referringto FIGS. 3A-3C, the image data reception unit 110 may receive CBCTimages for 3A a horizontal plane image, 3B a coronal plane image, and 3Ca sagittal plane image.

The multi-region segmentation unit 120 obtains a contrast histogram forthe contrast of the 3D medical image data, and segments the 3D medicalimage data into multiple regions based on the contrast histogram.

More specifically, the multi-region segmentation unit 120 divides arange of the contrast into 256 levels as shown in FIG. 4A, andcalculates the number of pixels having the contrast corresponding toeach level from the 3D medical image data, and thereby obtains thecontrast histogram. Next, the multi-region segmentation unit 120extracts, from the contrast histogram, a partial region having a peaksatisfying a specific criterion, that is, a meaningful peak using anAGMC (adaptive global maximum clustering) technique. For example,referring to FIG. 4B, four partial regions having colors such as red,blue, yellow green, and blue sky may be extracted from the contrasthistogram of FIG. 4A.

The multi-region segmentation unit 120 may segment the 3D medical imagedata into multiple regions based on an average value of the contrasthistogram of each partial region, and referring to FIG. 5, the 3Dmedical image data segmented into the multiple regions as shown in FIG.5B may be obtained from an original image that is FIG. 5A through themulti-region segmentation unit 120.

The skin detailed segmentation module of the purpose region segmentationunit 130 extracts a skin region by morphologically processing themultiple regions. Hereinafter, a skin detailed segmentation processusing a morphological technique which is performed in the purpose regionsegmentation unit 130 will be described with reference to FIG. 6.

<Skin Detailed Segmentation—Morphological Technique>

{circle around (1)} As shown in (a) of FIG. 6, as multi-segmentationresults of 3D medical image data, a given region may be labeled and aface candidate region may be obtained by binarizing the given region onthe basis of a label value, as shown in (b) of FIG. 6. For example, thegiven regions as the multi-segmentation results may be labeled as valuesof 1, 2, 3, and 4, and regions having a label value of 3 or larger maybe all binarized into a white color, and the remaining regions may beall binarized into a black color.

{circle around (2)} In order to remove a portion that does notcorrespond to an actual facial region from the binarized image, anerosion image as shown in (c) of FIG. 6 may be obtained by eroding animage using a circular structural element. For example, as shown in (b)of FIG. 6, a tool which is used when measuring the corresponding imageas well as the actual facial region may be also observed as a white thinstrip shape on the both sides from the erosion image, so that theerosion image may be obtained using the circular structural elementhaving a radius value set based on the size of the face candidate regionor a preset radius value in order to remove the white thin stripe shape.Here, other shapes other than the circular shape may be possible as theshape of the structural element. Through this, there is an effectcapable of removing unnecessary elements thinly connected to the face.

{circle around (3)} In order to further remove elements other than theface, which are not completely removed from the erosion image, an imageof a restricted erosion region as shown in (e) of FIG. 6 may be obtainedby applying a region restriction method as shown in (d) of FIG. 6. Here,the region restriction method may be applied according to the positionor distribution of the face candidate region in the corresponding imageand a region restriction value may be set. For example, as shown in (d)of FIG. 6, the face candidate region is distributed in the center of theimage, and therefore an erosion region is restricted from the center ofan X-axis of the image up to a place separated away in both directionsby ¼ (a region restriction value) of the total width of the image sothat the restricted erosion region may be obtained as shown in (e) ofFIG. 6.

{circle around (4)} The actual facial region may be obtained byextracting only connection components having a common portion with therestricted erosion region from the erosion image. That is, the actualfacial region as shown in (f) of FIG. 6 is obtained by extracting onlyconnection components having a common portion with (e) of FIG. 6 from(c) of FIG. 6. From (f) of FIG. 6, it can be seen that only the facialregion from which a tool which is used when measuring the correspondingimage and appears on the left and right sides of (c) of FIG. 6 isremoved is extracted.

{circle around (5)} The extracted facial region may have a reduced shapecompared to a face surface of an original image since it is obtainedfrom the erosion image eroded in step {circle around (2)}, so that inorder to restore this, an expansion operation may be performed using thesame structural element used in step {circle around (2)}, and then theinside of the erosion image may be filled to extract only the facesurface as shown in (g) of FIG. 6.

{circle around (6)} In order to smooth the boundary of the facialregion, an extracted skin may be obtained as shown in (h) of FIG. 6 byapplying Gaussian smoothing to (g) of FIG. 6. For example, Gaussiansmoothing that satisfies σ=2 may be applied.

Hereinafter, an erosion operation and an expansion operation of steps{circle around (2)} and {circle around (5)} will be described in moredetail with reference to FIG. 7.

Erosion and Expansion Operations *

When the center of the structural element is positioned in the boundaryof the original image along the boundary of the original image, it maybe classified into the erosion operation and the expansion operationaccording to the following cases.

Remove a portion in which the intersection exists: erosion operation

Add a portion in which the structural element is out of the originalimage: expansion operation

When the erosion operation and the expansion operation are sequentiallyperformed using such a structural element that is

, the size of the original image may be restored while regions smallerthan the corresponding structural element are removed, as shown fromFIG. 7A to FIG. 7E.

The mandible detailed segmentation module 132 of the purpose regionsegmentation unit 130 automatically extracts the mandible through atwo-dimensional (2D) detailed segmentation technique using an activecontour method based on a level set function with respect to each of 2Dimages.

3D medical image data may be represented as a set {I_(k)}_(k) ^(K)=1 of2D images as below, and the procedure for segmenting the mandible indetail may be divided into three steps as follows:

(i) Step of extracting mandible from sample image k_(s)

I_(k) _(s) : sample image→u_(k) _(s) : mandibular image

(ii) Step of applying 2D detailed segmentation technique to each imagein forward direction (k_(s)+1→K)

I_(k) _(s) +1 (k_(s)+1)-th image→u_(k) _(s) +1 (k_(s)+1)-th mandibularimage

I_(k) _(s) +2 (k_(s)+2)-th image→u_(k) _(s) +2 (k_(s)+2)-th mandibularimage

. . .

I_(K) K-th image→u_(K) K-th mandibular image

(iii) Step of applying 2D detailed segmentation technique to each imagein backward direction (k_(s)−1→1)

I_(k) _(s) −1 (k_(s)−1)-th image→u_(k) _(s) −1 (k_(s)−1)-th mandibularimage

I_(k) _(s) −2 (k_(s)−2)-th image→u_(k) _(s) −2 (k_(s)−2)-th mandibularimage

. . .

I₁ 1st image→u₁ 1st mandibular image

Hereinafter, each step will be described in detail with reference toFIGS. 8 to 10.

(i) Step of extracting mandible from sample image k_(s)

Referring to FIG. 8, a sample image is first selected (for example,k_(s)=60) (see (a) of FIG. 8), and regions having a label value of 4 orlarger are selected from a labeling image of the sample image (see (b)of FIG. 8). In order to remove a bone portion other than the mandible,only an upper portion of a center portion of a y-axis of thecorresponding image is selected (see (c) of FIG. 8). Only a connectioncomponent having a common portion with the bone portion is extractedfrom a restricted region (see (d) of FIG. 8). The mandible u_(k) _(s)for the finally extracted sample image is shown as a red line of (e) ofFIG. 8.

(ii) Step of applying 2D detailed segmentation technique to each imagein forward direction (k_(s)+1→K)

{circle around (1)} For the unity of a range on contrasts of all images,a range of a contrast of a given image is scaled into [0, 1].

{circle around (2)} An initial level set function for (k_(s)+1)-th imagesegmentation is defined as u_(k) _(s) ₊₁ ⁰=u_(k) _(s) +C (C≧0) usingu_(k) _(s) that is a segmentation result of the sample image. Forexample, C=2 may be satisfied, and adding a constant C may mean that acontour that is expanded by C in an outward direction from a contourindicating a previous mandible result is used as an initial contour fora current image. Since a thickness between upper and lower images issmall in the 3D medical image data, the position of a target within eachimage is not significantly changed, so that it may make initial contourautomatic prediction possible through expansion and contraction of theprevious result. FIG. 9 is an example of initial contour setting, and inFIG. 9, a red line indicates a segmentation result of a previous imageand a blue line indicates an initial contour of a current image and hasa distance difference by C with the segmentation result of the previousimage.

{circle around (3)} By applying the 2D detailed segmentation techniqueusing an (k_(s)+1)-th image I_(k) _(s) ₊₁ and an initial level setfunction u_(k) _(s) ₊₁ ⁰, a (k_(s)+1)-th mandibular image u_(k) _(s) ₊₁that is a final level set function that represents the mandible as0-level as shown in FIG. 10 may be obtained.

{circle around (4)} An initial level set function for a (k_(s)+2)-thimage I_(k) _(s) ₊₂ is defined as u_(k) _(s) ₊₂ ⁰=u_(k) _(s) ₊₁+C in thesame manner as in step {circle around (2)}, and a (k_(s)+2)-thmandibular image u_(k) _(s) ₊₂ may be obtained by performing the 2Ddetailed segmentation technique in the same manner as in step {circlearound (3)}. That is, by performing steps {circle around (2)} and{circle around (3)} on the following image, each mandibular image may besequentially obtained. The corresponding detailed segmentation isrepeated with an increment up to a K-th image, and automatically stopsat the number of an image in which the initial contour disappears.

(iii) Step of applying 2D detailed segmentation technique to each imagein backward direction (k_(s)−1→1)

In step (ii), detailed segmentation has been performed in a direction ofan increase in a value of k, and in step (iii), detailed segmentation isperformed in a direction of a reduction of the value of k to obtain eachmandibular image by performing the process of step (ii). At this point,the detailed segmentation is repeated with a decrement up to a (k=1)-thimage and automatically stops at the number of an image in which theinitial contour disappears.

Hereinafter, the 2D detailed segmentation technique used in steps (ii)and (iii) will be described in more detail with reference to FIGS. 11 to16.

<2D Detailed Segmentation Technique>

The 2D detailed segmentation technique is a useful method in a case inwhich a target within an image simultaneously has clear boundaries andblurry boundaries due to co-existence of bright and dark contrasts. Thisis a method (see FIG. 11) which makes an initial contour stop when theinitial contour reaches the boundary of the target by moving the initialcontour, as an active contour based-segmentation method. A level setmethod is used so as to make it free in phase changes of the initialcontour and a target contour, and in the level set method, the movementof the corresponding contour is described through a level set functionby introducing a higher-level function (level set function) capable ofrepresenting the corresponding contour as 0-level set (see FIG. 12). Asshown in FIG. 13, the level set method has an advantage capable ofdescribing various phases from a single level set function.

When I:

⊂

²→

is a given image, the corresponding contour may be described in a levelset form as follows:

$\begin{matrix}{{\phi_{0}(X)} = \left\{ {\begin{matrix}{d\left( {X,C_{0}} \right)} & {{inside}\left( C_{0} \right)} \\{- {d\left( {X,C_{0}} \right)}} & {{outside}\left( C_{0} \right)}\end{matrix},{C_{0} = {\left\{ {{Y\text{:}\mspace{14mu}{\phi_{0}(Y)}} = 0} \right\}\mspace{14mu}\underset{\_}{is}\mspace{14mu}{an}\mspace{14mu}{initial}\mspace{14mu}{{contour}.}}}} \right.} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$

Here, d(X,Y)=√{square root over ((x₁−y₁)²+(x₂−y₂)²)}, for X=(x₁,x₂) andY=(y₁,y₂) denotes an Euclidean distance, and therefore Equation 1becomes a signed distance function. In the segmentation method using thelevel set function, a final level set function that represents theboundary of the target as 0-level may be obtained by deforming the levelset function using a defined force in order to segment such an initiallevel set function. The purpose of detailed segmentation using the levelset function in the present invention is to properly find blurryboundaries as well as clear boundaries when the target simultaneouslyhas clear boundaries and blurry boundaries as shown in FIG. 14. In orderto find blurry boundaries, investigation should be sufficiently carriedout without passing by desired boundaries by controlling the movementspeed of the corresponding contour. Thus, the level set function isdeformed along an approximated delta function δ_(ε) which has a valueclose to 1 as it is closer to the contour and has a value close to 0 asit moves away from the contour while having a value other than 0 only inthe vicinity of ε- of the contour (0-level of the level set function).

$\begin{matrix}{{{\frac{\partial\phi}{\partial t}(X)} = {{F\left( {\phi(X)} \right)}{\delta_{ɛ}\left( {\phi(X)} \right)}}},{{\delta_{ɛ}\left( {\phi(X)} \right)} = \left\{ {\begin{matrix}\frac{1 + {\cos\left( \frac{{\pi\phi}(X)}{ɛ} \right)}}{2ɛ} & {{{if}\mspace{14mu}{{\phi(X)}}} < ɛ} \\0 & {otherwise}\end{matrix}.} \right.}} & \left\lbrack {{Equation}\mspace{14mu} 2} \right\rbrack\end{matrix}$

Here, F(φ(X)) denotes a force for deforming φ for segmentation, and willbe specified later.

The detailed segmentation technique may be developed through thefollowing two steps: (1) contrast emphasized image of the target and (2)equation for movement of contour.

(1) Contrast Emphasized Image of Target

Since a thickness of a z-axis is not large when 3D medical image data isobtained, the position of the target between upper and lower 2D imagesis not significantly changed. Accordingly, when using segmentationresult information of the previous image in order to set an initialcontour of a current image, the initial contour of the current image maybe set to include contrast information of the target while it ispositioned in the vicinity of the target. This assists in theimprovement of a segmentation speed and an efficient segmentationprocess.

First, a function for calculating an averaging contrast of a given imagein a region having a positive function value is defined as below.

$\begin{matrix}{{m(\phi)} = {\frac{1}{\int_{\Omega}^{\;}{{H\left( {\phi(X)} \right)}\ d\; X}}{\int_{\Omega}{{H\left( {\phi(X)} \right)}{I(X)}\ d\;{X.}}}}} & \left\lbrack {{Equation}\mspace{14mu} 3} \right\rbrack\end{matrix}$

Here, H(•) denotes a Heaviside function which has 1 with respect to apositive variable and has 0 with respect to a negative variable.

Next, a function for emphasizing an image is defined.J(X)=G _(σ) _(I) *exp(−r(I(X)−M)²), G _(σ) _(I) :Gaussian kernel, σ_(J),r≧0  [Equation 4]

This function has a value close to 1 when a contrast of a single pixelhas a value approximated to M indicating the contrast of the target, andhas a value close to 0 when the contrast of the single pixel has a valuethat is not approximated to M. This results in that the correspondingimage appears bright when the contrast of the single pixel has a valuesimilar to M and appears dark when the contrast of the single pixel hasa value other than M. When M indicates the contrast of the target, J maybe an image that emphasizes the contrast of the target. Accordingly, arelationship between the initial contour and the contrast of the targetis roughly divided into three cases as below, and then Equation 3 isused. Here, M indicates the contrast of the target.

(i) A case in which an averaging contrast of the inside of the initialcontour indicates the contrast of the target,M=m(φ₀)

(ii) A case in which a bright contrast of the inside of the initialcontour indicates the contrast of the targetM=m(I−m(φ₀))

(iii) A case in which a dark contrast of the inside of the initialcontour indicates the contrast of the targetM=m(−I+m(φ₀))

In Equation 4, when a constant r is significantly large, only pixelshaving a contrast significantly similar to the contrast of the targetmay appear bright and most of pixels may appear dark, and when theconstant r is significantly small, pixels having a contrast evenslightly similar to the contrast of the target may appear bright. Thatis, r may determine the sensitivity concerning a difference with atarget contrast, and an image for properly emphasizing a target imagemay be obtained by obtaining an appropriate value. For example, in acase of r=10, (ii) may be used, and in FIG. 15, a process of acquiring acontrast emphasized image of the target using (ii) is shown.

(2) Equation for Movement of Contour

Using the above-defined emphasized image, an evolution equation for aforce and a level set function which are involved in the movement of thecontour is derived. In order to properly find blurry boundaries of thetarget, a force that represents local properties of each pixel should bedefined. Equation 5 represents a local force that is defined using adifference between an averaging contrast of the inside and outside ofthe contour which has been locally calculated in the vicinity of thepixel for the emphasized image and a contrast of the pixel itself.

$\begin{matrix}{{{F_{l}\left( {{J(X)},{\phi(X)}} \right)} = {{- \left\lbrack {{J(X)} - {f_{1}(X)}} \right\rbrack^{2}} + \left\lbrack {{J(X)} - {f_{2}(X)}} \right\rbrack^{2}}}{{{f_{1}(X)} = \frac{\left\lbrack {K_{A}*\left( {{H_{ɛ}(\phi)}J} \right)} \right\rbrack(X)}{\left\lbrack {K_{A}*{H_{ɛ}(\phi)}} \right\rbrack(X)}},{{f_{2}(X)} = \frac{\left\lbrack {K_{A}*\left( {{H_{ɛ}\left( {- \phi} \right)}J} \right)} \right\rbrack(X)}{\left\lbrack {K_{A}*{H_{ɛ}\left( {- \phi} \right)}} \right\rbrack(X)}}}} & \left\lbrack {{Equation}\mspace{14mu} 5} \right\rbrack\end{matrix}$

Here, K_(h) denotes an averaging kernel having a kernel size of 2h+1,and determines a degree of local calculation. A function H_(ε) uses anintegrand of the approximated delta function used in Equation 2 as anapproximated version of a Heaviside function.

${H_{ɛ}\left( {\phi(X)} \right)} = \left\{ {\begin{matrix}{\frac{1}{2}\left( {1 + \frac{\phi(X)}{ɛ} + {\frac{1}{\pi}{\sin\left( \frac{{\pi\phi}(X)}{ɛ} \right)}}} \right)} & {{{if}\mspace{14mu}{{\phi(X)}}} < ɛ} \\1 & {{{if}\mspace{14mu}{\phi(X)}} \geq ɛ} \\0 & {{{if}\mspace{14mu}{\phi(X)}} \leq {- ɛ}}\end{matrix}.} \right.$

The following Equation 6 is an equation that represents a curvature ofthe corresponding contour and serves to keep an irregular shape of thecontour more flat, thereby helping to prevent the deviation of thecontour from desired boundaries.

$\begin{matrix}{{F_{r}\left( {\phi(X)} \right)} = {\nabla{\cdot {\left( \frac{\nabla{\phi(X)}}{{\nabla{\phi(X)}}} \right).}}}} & \left\lbrack {{Equation}\mspace{14mu} 6} \right\rbrack\end{matrix}$

Finally, an evolution equation for detailed segmentation is obtained asshown in Equation 7.

                                 [Equation  7]${{\frac{\partial\phi}{\partial t}(X)} = {\left\lbrack {{\lambda\;{F_{l}\left( {J,\phi} \right)}} + {\mu\;{F_{r}(\phi)}} + {\gamma\;{g\left( {I(X)} \right)}}} \right\rbrack{\delta_{ɛ}\left( {\phi(X)} \right)}}},\gamma,\lambda,{\mu \geq 0.}$

Here, a function g is a function that depends on an original image I,and defined as follows:

${{g\left( {{\nabla{I(X)}}} \right)} = \frac{1}{1 + {\tau{{{\nabla\left( {G_{\sigma}*I} \right)}(X)}}^{2}}}},{\tau \geq 0},$

Since the function g has a value close to 0 on edges and has a valueclose to 1 in smooth regions other than the edges, it is called an edgeindicator function (see FIG. 16A and FIG. 16B). When τ is increased, thefunction g is sensitive to even small changes in the contrast andthereby represents most of pixels as the edges. For the appropriate useof τ, only desired edges may be expressed as dark portions. For example,τ=1000 may be used. A value of F_(l) that is a local force may becomesignificantly small in the smooth regions so that evolution of the levelset may stop in undesired regions, which may lead to a wrong result. Theedge indicator function g has a large value only in the smooth regions,and therefore it is possible to prevent the contour from stopping in theundesired regions by inducing the movement of the contour in the smoothregions.

Consequently, a first term ,λF_(l)(J,φ) of Equation 7 denotes a localforce that prevailingly exerts so as to evolve the level set functionthrough a contrast comparison between the inside and outside of thecontour which has been locally calculated, a second term μF_(r)(φ)thereof serves to make the contour flat, and a third term γg(I(X))thereof is introduced to prevent a stopping phenomenon of the contourdue to the local force that becomes significantly small in the smoothregions. In addition, δ_(ε)(φ(X)) helps to carefully investigate theblurry boundaries by limiting a change region of the level set to thevicinity of ε- of 0-level. When the right side of Equation 7 approaches0 so that there is no change in ,φ, 0-level set of ,φ represents a finalsegmentation result. FIG. 17 shows a detailed segmentation process usingthe level set function of Equation 7.

The 3D reconstruction unit 140 reconstructs the skin region and mandibleextracted by the purpose region segmentation unit 130 as a 3D model.

A level set function ,{u_(k)}_(k=1) ^(K) that represents the mandible as0-level may be obtained through the purpose region segmentation unit130. A 0-level iso-surface of this data may be reconstructed as the 3Dmodel of the mandible using a marching cube algorithm. At this point, inorder to eliminate a staircase phenomenon caused by a voxel size, thesurface is subjected to smoothing treatment using the HC-Laplacianalgorithm (see FIG. 18).

In addition, FIGS. 19 and 20 show the superiority of the presentinvention compared to the conventional method.

The conventional method has the result obtained through a variety ofpost-processing after initial segmentation through iso-value adjustmentusing the Mimics software, and such a method has difficulties when atarget having irregular boundaries is segmented. In the conventionalmethod, the iso-value may be adjusted to be low in order to extract theblurry boundary, so that outer peripheral portions of the boundary of adesired target as well as even unnecessary portions around the blurryboundaries are likely to be extracted. On the other hand, the methodproposed in the present invention may find only the boundary of thetarget in an appropriate position.

The control unit 150 controls operations of the image data receptionunit 110, the multi-region segmentation unit 120, the purpose regionsegmentation unit 130, and the 3D reconstruction unit 140, and controlsthe flow of data thereof.

FIG. 2 is a flowchart illustrating a method for forming a 3Dmaxillofacial model by automatic medical image segmentation, which isperformed in the automatic image segmentation and model formation serverof FIG. 1.

In operation 5210, the image data reception unit 110 receives 3D medicalimage data that is a set of 2D images. Here, the 3D medical image datamay be a CBCT image.

In operation S220, the multi-region segmentation unit 120 obtains thecontrast histogram for a contrast of the 3D medical image data, andsegments the 3D medical image data into multiple regions based on thecontrast histogram. Here, the multiple regions may be labeled accordingto the contrast. More specifically, the multi-region segmentation unit120 divides a range of the contrast into 256 levels, and calculates thenumber of pixels having the contrast corresponding to each level fromthe 3D medical image data, and thereby obtains the contrast histogram.Next, a partial region having a peak satisfying a specific criterion isextracted using an AGMC (adaptive global maximum clustering) techniquefrom the contrast histogram, and the 3D medical image data is segmentedinto the multiple regions based on an average value of the contrasthistogram of the partial region.

In operation S230, the skin detailed segmentation module 131 of thepurpose region segmentation unit 130 extracts a skin region bymorphologically processing the multiple regions. More specifically, theskin detailed segmentation module 131 obtains a face candidate region bybinarizing the multiple regions according to label values of themultiple regions, and erodes the face candidate region using a circularstructural element having a radius value set based on a size of the facecandidate region or a preset radius value. Next, the skin detailedsegmentation module 131 restricts the eroded region of the facecandidate region, extracts a connection component having a commonportion with the eroded region, and then expands the eroded region againusing the circular structural element.

In operation S240, the mandible detailed segmentation module 132 of thepurpose region segmentation unit 130 automatically extracts the mandiblethrough a 2D detailed segmentation technique using a level setfunction-based active contour method with respect to each of 2D images.More specifically, the mandible detailed segmentation module 132extracts a segmentation result for a sample image among the 2D images,and extracts all of segmentation results for the remaining images usingthe segmentation result for the sample image. The segmentation resultfor each image is extracted through a process of setting the initialcontour, emphasizing the contrast, and moving the contour.

In operation S250, the 3D reconstruction unit 140 reconstructs the skinregion and the mandible extracted by the purpose region segmentationunit 130, as a 3D model. More specifically, the 3D reconstruction unit140 reconstructs the skin region and the mandible as the 3D model usinga surface rendering algorithm, and makes the surface of thereconstructed 3D model smooth using an HC-Laplacian algorithm

As described above, according to the present invention, the skin andmandible of a patient may be automatically segmented and a 3D model maybe generated by combining a multi-segmentation method and a detailedsegmentation method using a level set function, thereby helping toestablish more accurate and efficient surgical plan and obtain a uniformmandibular model of the patient. In other words, there is an effectcapable of more efficiently establishing a surgical plan with less time.

In addition, through a 2D detailed segmentation technique, the methodaccording to the present invention may be performed at a high speedwhile overcoming difficulties in the segmentation due to irregularitiesin the boundaries caused by high and low contrasts.

In addition, the mandible portion may be segmented by cross-sections in2D medical images, and then, the segmentation results, that is,mandibular images extracted from each of the 2D medical images may bestacked to form a 3D model, thereby obtaining a more accurate 3D modelof the mandible.

It should be apparent to those skilled in the art that variousmodifications can be made to the above-described exemplary embodimentsof the present invention without departing from the spirit or scope ofthe invention. Thus, the present invention is intended to cover all suchmodifications provided they come within the scope of the appended claimsand their equivalents.

What is claimed is:
 1. A method for forming a three-dimensional (3D)model of skin and mandible by automatic medical image segmentation whichis performed in an automatic image segmentation and model formationserver, the method comprising: (a) receiving 3D medical image data thatis a set of two-dimensional (2D) images for horizontal planes of a face;(b) obtaining a contrast histogram based on distribution of contrasts ofthe 3D medical image data, and segmenting the 3D medical image data forthe face into multiple regions separated into at least one partialregion based on the contrast histogram; (c) extracting only the face byremoving portions other than the face from the multiple regions for theface, and extracting a skin region of the face; (d) extracting themandible from each of the 2D images for the horizontal planes of theface through a 2D detailed segmentation technique using an activecontour method based on a level set function; and (e) reconstructing theextracted skin region and mandible as the 3D model.
 2. The method forforming the 3D model of claim 1, wherein the (b) obtaining of thecontrast histogram includes dividing a range of the contrast intopredetermined levels, and calculating the number of pixels having thecontrast corresponding to each of the levels from the 3D medical imagedata for the face to thereby obtain the contrast histogram.
 3. Themethod for forming the 3D model of claim 2, wherein the (b) obtaining ofthe contrast histogram further includes extracting, from the contrasthistogram, a partial region having a peak satisfying a specificcriterion using an AGMC (adaptive global maximum clustering) technique.4. The method for forming the 3D model of claim 3, wherein the (b)obtaining of the contrast histogram further includes segmenting the 3Dmedical image data into the multiple regions based on an average valueof the contrast histogram of the partial region.
 5. The method forforming the 3D model of claim 1, wherein the (c) extracting of only theface and the skin region thereof includes obtaining a face candidateregion by binarizing the multiple regions according to label values ofthe multiple regions.
 6. The method for forming the 3D model of claim 5,wherein the (c) extracting of only the face and the skin region thereoffurther includes eroding the face candidate region using a circularstructural element having a radius value set based on a size of the facecandidate region or a preset radius value.
 7. The method for forming the3D model of claim 6, wherein the (c) extracting of only the face and theskin region thereof further includes restricting an erosion regionreaching from a center of the face candidate region to positionslaterally away from each other by a preset length.
 8. The method forforming the 3D model of claim 7, wherein the (c) extracting of only theface and the skin region thereof further includes extracting aconnection component having a common portion with the erosion region,and expanding the erosion region using the circular structural element.9. The method for forming the 3D model of claim 1, wherein the (d)extracting of the mandible includes selecting a sample image from the 2Dimages, and extracting the mandible from the sample image that is asegmentation result for the sample image.
 10. The method for forming the3D model of claim 9, wherein the (d) extracting of the mandible furtherincludes setting an initial contour of an image next to or prior to thesample image based on a contour for a segmentation result of the sampleimage.
 11. The method for forming the 3D model of claim 10, wherein the(d) extracting of the mandible further includes emphasizing a contrastof the mandible based on contrast information about the mandible insidethe initial contour.
 12. The method for forming the 3D model of claim11, wherein the (d) extracting of the mandible further includes stoppingmovement of the contour when the initial contour moves and reaches aboundary of the mandible.
 13. The method for forming the 3D model ofclaim 12, wherein the (d) extracting of the mandible further includesthe contour moving based on an average value of local contrasts of theinside and outside of the contour and a curvature of the contour. 14.The method for forming the 3D model of claim 1, wherein the (e)reconstructing of the extracted skin region and mandible includesreconstructing the extracted skin region and mandible as the 3D modelusing a surface rendering algorithm, and processing a surface of the 3Dmodel using an HC-Laplacian algorithm.
 15. The method for forming the 3Dmodel of claim 1, wherein the 3D medical image data is CBCT (cone beamcomputed tomography) image data.
 16. The method for forming the 3D modelof claim 1, wherein the segmented multiple regions are labeled accordingto contrasts of the multiple regions.
 17. An automatic imagesegmentation and model formation server which performs a method forforming a 3D model of skin and mandible by automatic medical imagesegmentation, the server comprising: an image data reception unit thatreceives 3D medical image data that is a set of 2D images for horizontalplanes of a face; a multi-region segmentation unit that obtains acontrast histogram based on distribution of contrasts of the 3D medicalimage data, and segments the 3D medical image data for the face intomultiple regions separated into at least one partial region based on thecontrast histogram; a purpose region segmentation unit that includes askin detailed segmentation module for extracting only the face byremoving portions other than the face from the multiple regions for theface and extracting a skin region of the face, and a mandible detailedsegmentation module for extracting the mandible from each of the 2Dimages for the horizontal planes of the face through a 2D detailedsegmentation technique using an active contour method based on a levelset function; and a 3D reconstruction unit that reconstructs theextracted skin region and mandible as the 3D model.
 18. The automaticimage segmentation and model formation server of claim 17, wherein themulti-region segmentation unit obtains the contrast histogram about the3D medical image data, and segments the 3D medical image data into themultiple regions based on an average value of the contrast histogram ofa partial region having a peak satisfying a specific criterion in thecontrast histogram.
 19. The automatic image segmentation and modelformation server of claim 17, wherein the skin detailed segmentationmodule obtains a face candidate region from the multiple regions, erodesthe face candidate region using a circular structural element based on asize of the face candidate region, restricts an erosion region based ona length of the face candidate region, extracts a connection componenthaving a common portion with the erosion region, and expands the erosionregion using the circular structural element.
 20. The automatic imagesegmentation and model formation server of claim 17, wherein themandible detailed segmentation module segments in detail an image nextto or prior to a sample image based on a contour for a segmentationresult of the sample image among the 2D images to thereby extract themandible.
 21. A non-transitory computer-readable recording medium thatrecords a program capable of executing the method of claim 1 using acomputer.